Fingerprint recognition system

ABSTRACT

A method and apparatus for fingerprint recognition is provided. A fingerprint is detected on a sensor. The fingerprint is digitized. A digitized background is subtracted from the fingerprint, resulting in a difference print. The difference print is matched with a database of fingerprint templates.

FIELD OF THE INVENTION

[0001] The present invention relates to biometrics, and morespecifically, to electronic verification and identification ofindividuals using biometrics.

BACKGROUND OF THE INVENTION

[0002] Securing computer systems and electronic transactions is becomingmore and more important as we enter the electronic age. Existingpassword and cryptographic techniques seem well on their way to solvingthe security problems of computer systems, electronic commerce, andelectronic transactions. These solutions ensure that the set of digitalidentification keys associated with an individual person can safelycarry on electronic transactions and information exchanges. Little,however, has been done to ensure that such identification keys can onlybe used by their legitimate owners. This is a critical link that needsto be made secure if secure computer access, electronic commerce, homebanking, point of sale, electronic transactions, and similar mechanismsare to become truly secure.

[0003] Today, passwords handle most of these issues. For example, mostelectronic transactions, such as logging into computer systems, gettingmoney out of automatic teller machines, processing debit cards,electronic banking, and similar transactions require passwords.Passwords are an imperfect solution because as more and more systemsattempt to become secure, a user is required to memorize an everexpanding list of passwords. Additionally, passwords are relativelyeasily obtained by observing an individual when he or she is enteringthe password. Moreover, there is no guarantee that users will notcommunicate passwords to one another, lose passwords, or have themstolen. Thus, passwords are not considered sufficiently secure for manyfunctions.

[0004] More and more often, fingerprint identification is considered.Fingerprints have the advantage of being unique to an individual person,requiring no memorization, and being relatively difficult toappropriate. Thus, some secure systems are switching to fingerprintrecognition. Fingerprint recognition generally requires a user to placehis or her finger on a fingerprint sensing device. Each fingerprintconsists of a unique arrangement of ridges and grooves. The fingerprintsensing device transmits an analog image of the user's fingerprint, viaa coaxial cable, to a computer system. The computer system then matchesthe fingerprint to a database of fingerprint templates in the computersystem. However, there are a number of problems with prior artfingerprint identification methods.

[0005] First, the fingerprint sensing devices, the devices on which thefinger is placed, are generally bulky. This means that such devices cannot be adapted to be used with portable computers, consumer electronics,or in situations where space is precious.

[0006] Additionally, fingerprint devices generally require a connectionto a power outlet in addition to the connection to the computer system.This means that if the fingerprint device needs to be used, anadditional power outlet is consumed. Thus, such devices can not beadapted for use in situations where extra power outlets are notavailable.

[0007] Furthermore, because conventional fingerprint devices generallytransmit an analog image of the fingerprint, via a coaxial cable,security may be breached. The analog fingerprint image may be obtainedby intercepting the image transmitted on the coaxial cable. Thus, if afalse user had an image capturing device, he or she may be able toimpersonate an original user, by resending a captured image. Thisreduces security in cases where the actual fingerprint sensing procedureis not observed by anyone.

[0008] Furthermore, because fingerprint processing in conventionalsystems generally takes place in the computer system, the computersystem itself can be corrupted to defeat the security provided by thefingerprint sensing device. In the end, the computer system decideswhether the fingerprint received from the device matches a print in thedatabase. Either the database can be altered or the process whichmatches the print to the database can be altered to send a falsepositive indication. In this way, the advantages of the fingerprintsensing system may be lost.

[0009] Moreover, in conventional systems the user is required tointeract with the fingerprint sensing system. Generally, the prior artprocess of sensing a fingerprint is as follows. First, the userpositions his or her finger on the sensing sensor platen. An image ofthe fingerprint is displayed on the computer monitor, with a cross-hair.The user is asked to position his or her finger such that the crosshairs are centered, and that the print is clearly displayed. When theuser has determined that the finger is in the proper position, the usermust press a button to indicate that this is the image to betransmitted. Once the user has selected the proper fingerprint, thedevice takes an image, and sends it to the computer system forprocessing. However, this awkward and error-prone procedure requiresactive participation and

BRIEF SUMMARY OF THE INVENTION

[0010] The method and apparatus for fingerprint recognition is provided.A fingerprint is detected on a sensor. The fingerprint is digitized. Adigitized background is subtracted from the fingerprint, resulting in adifference print. The difference print is matched with a database offingerprint templates.

[0011] The background is obtained by taking an image of the sensorplaten at the time of startup, and after each fingerprint is detected.Thus, the current state of the background, including smudges, lightingunevenness, and other factors are detected and subtracted from thedigitized fingerprint.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present invention is illustrated by way of example, and notby way of limitation, in the figures of the accompanying drawings and inwhich like reference numerals refer to similar elements and in which:

[0013]FIG. 1 illustrates an overview level diagram of a sample system inwhich one embodiment of the present invention may be implemented.

[0014]FIG. 2 is a functional block diagram of a digital processingsystem and a sensor.

[0015]FIG. 3 is a functional block diagram of one embodiment of anetworked sensor and server.

[0016]FIG. 4 is a functional block diagram of one embodiment of awallet.

[0017]FIG. 5 is an overview flowchart illustrating the process occurringin the sensor.

[0018]FIGS. 6A and 6B are an overview flowchart illustrating the processoccurring in the digital system.

[0019]FIG. 7 is a flowchart illustrating one embodiment of the securityprocess in the present invention.

[0020]FIGS. 8A and 8B are a diagram of one embodiment of the sensor ofthe present invention.

[0021]FIG. 9 is a diagram of the FPGA of FIG. 8.

[0022]FIG. 10 is a diagram of one embodiment of the digital system ofthe present invention.

[0023]FIGS. 11A and 11B are a flowchart illustrating the process ofregistering a fingerprint.

[0024]FIG. 12A is a flowchart illustrating the process of capturing afingerprint image.

[0025]FIG. 12B is an illustration of the filtering process used in thepresent invention.

[0026]FIG. 12C is an illustration of the fingerprint snapshots takenprogressively during the process of capturing the fingerprint image.

[0027]FIG. 13 is a flowchart illustrating the process of auto-launching.

[0028]FIG. 14 is a flowchart illustrating the process of using a smartcard in conjunction with the fingerprint recognition system of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0029] A method and apparatus for electronic verification andidentification of individuals using biometrics is described. In thefollowing description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding of the present invention. It will be apparent, however, toone skilled in the art that the present invention may be practicedwithout these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order to avoidunnecessarily obscuring the present invention.

[0030] Overview

[0031]FIG. 1 illustrates an overview level diagram of a sample system inwhich one embodiment of the present invention may be implemented. Adigital system 110 is a computing system which has the processingability to compare a received digitized image with a database ofdigitized templates, and control a digital connection for receiving thedigitized image. In the preferred embodiment, the digital connection isa data bus which conforms to a universal serial bus (USB) standard, asis well known to those of ordinary skill in the art. In this figuredigital system 110 is represented as a computer system. The computersystem 110 includes a body 120, which contains the processing power ofthe computer system 110. Computer system 110 also includes a display130. The display 130 may be a liquid crystal display (LCD), cathode raytube (CRT), or similar display mechanism. Computer system 110 includes adata entry mechanism 140. In this instance, a keyboard 140 isillustrated. The keyboard 140 permits a user to interact with thecomputer system 110. A conventional cursor control device 145 is furtherillustrated. The cursor control device 145 may be a mouse, trackball,pen, or similar device.

[0032] In one embodiment, sensor 150 is coupled to the computer system110 via a cable 170. Alternatively, sensor 150 may be coupled tocomputer system 110 via an infrared, radio frequency, modem, network, orany other direct or indirect digital connection.

[0033] The sensor 150 of the present invention includes a sensor platen160, on which a finger is placed for the fingerprint recognition. In oneembodiment, cable 170 is a universal serial bus (USB) connection. Itwill be apparent to those of ordinary skill in the art that otherdigital connections may also be used. The sensor 150 may further have aconnection to a power source. However, if the cable 170 is a USBconnection, no such additional power connection is required as the USBconnection provides power. It is to be understood that FIG. 1 is merelyan illustration of one embodiment of a system on which the presentinvention may be practiced. Alternate configurations, such as a portablecomputer 110, a digital system which does not have all of the componentsillustrated, or a sensor 150 having a different shape or size may alsobe utilized.

[0034]FIG. 2 is a functional block diagram of the digital system 210 andsensor 250 of the present invention. In one embodiment, the digitalsystem 210 may be embodied in a computer system 110. The digital system210 includes a temporary data storage 215, for storing data temporarily.The temporary data storage 215 may include random access memory (RAM),and various registers. Digital system 210 further includes database 220.Database 220 is for storing fingerprint templates, identification data,etc. for each individual person who is registered with that system.Comparator 225 is for comparing fingerprint data. In one embodiment, thecomparator 225 is able to compare data stored in the database 220 withdata stored in the temporary data storage 215. The comparator 225 has anoutput which determines whether or not the data provided to it match ornot.

[0035] Security unit 230 is utilized to encrypt and decrypt messagessent between the digital system 210 and the sensor 250 on line 290 andto determine, maintain and use session keys. Security unit 250 isfurther described below. An interface 235 interacts with the user andwith other programs in the digital system 210 and the sensor 250.Interface 235 may display various windows in a WINDOWS or MACINTOSHenvironment. Windows is a trademark of Microsoft Corporation, andMacintosh is a trademark of Apple Computers, Inc.

[0036] Card receiving unit 240 may be integral with digital system 210,or it may be attached to digital system 210 via a bus, cable, infrared,or other connection method. Card receiving unit 240 is for receiving atoken, smart card, barcode, diskette, or similar medium which may storepersonal information about the holder of the card, and may containfingerprint information. The card receiving unit 240 may be utilized toverify the identity of the card holder with respect to fingerprintinformation stored on the card.

[0037] Registering unit 245 enables a user to register with the digitalsystem 210, such that the user's fingerprint identification is placed inthe database 220. The digital system 210 may further include a universalserial bus (USB) controller 205. The universal serial bus controller 205couples the digital system 210 with the sensor 250, in one embodiment.Universal serial bus controller 205 provides a data conduit as well aspower to sensor 250. The functioning of the universal serial buscontroller 205 may be found in more detail in the Universal Serial BusSpecification, Revision 1.0, Jan. 15, 1996.

[0038] Sensor 250 is coupled to the digital system 210 throughconnection 290. In one embodiment, the connection 290 is USB, whichprovides both data and power connections. Alternatively, sensor 250 mayhave a separate power connection.

[0039] Sensor 250 includes a data storage unit 255. Data storage unit255 may include RAM, registers, as well as memory. Data storage unit 255stores intermediate values of prints, templates, sums, session keys,permanent sensor signature, and similar data.

[0040] Sensor 250 further includes a sensing mechanism 260. The sensingmechanism 260 may include a sensor platen, on which a user can place hisor her fingers for recognition. The sensing mechanism 260 may be aconventional fingerprint sensing mechanism, consisting of a light,illuminating at least one prism, which reflects the print on the sensorplaten. The reflected print is received by a detector array.Alternatively, sensing mechanism 260 may utilize other methods ofsensing, including capacitive sensors.

[0041] Sensor 250 further includes a digitizer 265. Digitizer 265digitizes images received from the sensing mechanism 260. Mechanismswhich may be used to digitize an image are known in the art. In oneembodiment, a conventional analog-to-digital converter is utilized.

[0042] Sensor 250 further includes a subtractor 270. Subtractor 270 isutilized to filter a digitized fingerprint image and subtract abackground image from a print, as will be described below.

[0043] Security unit 275 in the sensor 250 corresponds to the securityunit 230 in the digital system 210. However, it may further store theprivate key of the sensor, its signature, in a tamper-proof environment.

[0044] Finally, sensor 250 includes decision making unit 280. Decisionmaking unit 280 may be utilized to make a final determination whether afingerprint matches the print in the database 220. Decision making unit280 may be used when digital system 210 is not secure, and strictsecurity is necessary. The functioning of the above described componentsis elaborated further below.

[0045] In one embodiment, digital system 210 may be a computer system, aPCMCIA card, a portable computer, a network station and server, a palmtop computer, or any other system which may be capable of processing thedata required. Furthermore, the sensor 250 may be located within thedigital system 210. In such a case, no duplicative memory, securityunits and USB controller would be required.

[0046]FIG. 3 illustrates a network in which the present invention may beutilized. Sensor 310 is coupled to host 320. Host 320 is enabled toconnect to a network 330, which couples a plurality of systems 320, 340,350 together. A server 340 contains the database which is matched to thefingerprint received by sensor 310. Other systems 350 may be utilizedfor their processing power. Thus, the actual fingerprint recognitionprocess may be distributed over a plurality of systems 320, 340, 350.Such a distributed processing may be used for accessing remote datathrough a network. Because neither the server 340, nor the other systems350 are secure, for security purposes final matching may be done in thesensor 310. This would be accomplished by sending the processed databack to the sensor 310. Thus, the sensor 310 receives matched elements,and the original fingerprint. Verifying that the matched elements trulymatch the original fingerprint is a process which may be accomplished inthe sensor 310. Thus, the sensor 310 may send out the final matched/notmatched signal, thus creating a secure system over an insecure network330.

[0047]FIG. 4 illustrates a block diagram of one embodiment of a systemwhich combines the functionality of the sensor 250 and digital system210 into a single unit, called a wallet 400. A wallet 400 may beimplemented with different configurations of software and hardware. Forexample, the entire wallet 400 may reside in a smart card, or it may beimplemented as a distributed system that may include a smart card,database, and matching/control software distributed over a network.

[0048] Sensing unit 410 has a sensor platen 415 on which a finger isplaced. Sensing unit 410 receives the image, and passes it on todigitizer 420. Digitizer 420 digitizes the fingerprint image, and passesit on to a matching unit 425. Matching unit 425 further has access to astorage unit 430, which stores a database of templates. Matching unit425 matches the features of the received fingerprint to the templates inthe storage unit 430. In one embodiment, such a wallet 400 belongs toone individual only, whose print is stored in the storage unit 430. Thematching unit 425 passes on a yes/no decision, whether prints match, toa data flow control unit 435. The data flow control unit 435 controlsaccess to data stored in a user data unit 440. The data flow controlunit 435 may further allow the user to upload information to the wallet400 once the user's access to the wallet 400 is verified.

[0049] The user data unit 440 may contain such information as the user'scredit card number, social security number, and identity. The user dataunit 440 may further contain any information a user wishes to store inthe user data unit 440. The wallet 400 further may include a controlmechanism 445, such as a keyboard, mouse, trackball, touch pad, etc. Theuser may utilize the control mechanism 445 to add data to the wallet400.

[0050]FIG. 5 is a flowchart illustrating the overview of the processoccurring in the sensor 250. At block 505, the sensor is turned on. Thismay occur when the digital system 210 is first turned on. Alternatively,because the universal serial bus 290 permits plug-and-play, this mayoccur when the sensor is first connected to the digital system 210.

[0051] At block 510, initialization and a security handshake occurs.Because the connection between the digital system 210 and the sensor 250is designed to be a secure connection, a security handshake ensures thatthe sensor 250 has not been substituted, and that the digital system 210has not been tampered with. A security handshake may be implementedusing conventional encryption or public/private key methods.

[0052] At block 515, a background image is captured and digitized. Whenthe sensor 250 is initialized, the optical light source in the sensingunit 260 is turned on, and an image is captured. At this time, there isno fingerprint on the sensor platen, thus this image is a backgroundimage. The digitized background is temporarily stored in data storage255. Alternatively, the background image may be sent to the digitalsystem 210, and stored in the data storage unit 215. The backgroundimage may be further used to estimate the background noise and image,and estimate the overall background. In one embodiment, each backgroundimage captured is further processed to create an average background,which may be utilized by the sensor. This learning process will bedescribed in more detail below.

[0053] At block 520, the system tests whether a fingerprint is detectedon the sensor platen. In one embodiment, the fingerprint detection is ameasurement of the energy level of the light received from the sensorplaten. If the energy level drops below a certain level the fingerprintis detected. Alternatively, a fingerprint may be detected as a result ofa pressure sensor, a heat sensor, or a similar device.

[0054] If no fingerprint is detected on the sensor platen, the processreturns to block 515, and a new background image is captured anddigitized. In one embodiment, if no fingerprint is detected, a newbackground image is captured approximately every 2 seconds.Alternatively, the original background may be maintained. If afingerprint is detected, the process continues to block 525.

[0055] At block 525, the fingerprint image is captured and digitized. Asdescribed above, the sensing unit 250 receives an image of thefingerprint placed on the sensor platen which is captured by a detectorarray. The captured fingerprint is digitized, and stored. The digitizedprint may be stored in the sensor 250 or in the digital system 210.

[0056] At block 530, the digitized background image is subtracted fromthe digitized fingerprint, resulting in a differential print.Subtracting the background image from the fingerprint image decreasesthe level of image corruption due to smudges and dirt on the sensorplaten, or uneven lighting conditions. By subtracting the backgroundimage, the differential print is a clearer representation of thefingerprint which was placed on the sensor platen. Subtraction heremeans either taking the difference of the acquired print minus thebackground image, or any function of the image and background thatdecreases the effect of imperfections in the optics and the imagingconditions. In one embodiment, during subtraction, if the pixel value ofthe background is higher than the value of the fingerprint, the resultis set to a default value. Thus, for example, if the pixel value of thebackground is 150, and the pixel value of the fingerprint is 30, theresult may be set to 0, 30, or another default value. In this way, falsesubtractions and negative numbers are avoided.

[0057] At block 535, a new background image is captured and digitized.As the finger is removed from the sensor, the image of the newbackground, the result of the last finger's smudges added to theprevious background, is newly captured.

[0058] At block 540, a nonce is received from the digital system 210. Anonce is a signal or code used only for one particular occasion, and notreused. The nonce includes a date/time stamp, and possibly other data.

[0059] At block 545, a hash is created of the differential print. Thehash, sent with the print, makes sure that the matched print is in factthe print which was captured by the sensor 250. Additionally, byincluding the nonce in the hash, the time/date of the fingerprintmatched is verified. This prevents a user from resending an older printto obtain a false positive indication.

[0060] At block 550, the hash and differential print is sent to thedigital system 210. The digital system 210 is used to process thefingerprint further, as described in more detail below.

[0061]FIGS. 6A and 6B are an overview flowchart illustrating the processoccurring in the digital system 210. The digital system 210 may be acomputer system, a server coupled to a computer system, a self-containedPCMCIA card, a network, or similar device or devices which have theprocessing power to match fingerprints.

[0062] At block 605, the system is turned on and/or coupled to thesensor. In one embodiment, this occurs automatically when the computersystem or other hardware on which the digital system 210 is implementedis first turned on. Alternatively, a user may initiate the presentsystem by a keystroke, mouse selection, coupling the sensor 250 to thedigital system 210 via a plug-and-play connection, or similar action.

[0063] At block 610, the universal serial bus is initialized. Theuniversal serial bus is a communications architecture that provides adata and power connection. The universal serial bus controller 205 hasto be initialized according to the USB specification. Alternatively, ifthe connection is not a USB connection, this step may be disregarded oranother digital connection initialization step performed.

[0064] At block 615, an initialization signal and power is sent to thesensor 250. This turns on the sensor 250, as described above withrespect to FIG. 5. A protocol initializes the information which isnecessary for secure transactions. This process is described in moredetail below.

[0065] At block 620, it is determined whether a signal has been receivedthat a fingerprint has been detected by the sensor 250. When the sensor250 detects a fingerprint, it sends a signal to the digital system 210.If no such signal is received, the process waits for this signal. Whenthe signal is received, the process continues to block 625. Althoughthis process is illustrated as a loop in FIG. 6A, in one embodiment,this may be an interrupt driven process, in which the system may performother activities while “waiting”. The arrival of a signal from thesensor 250 generates an event (interrupt) in the digital system, bywhich appropriate software in the digital system 210 is activated, torespond to the signal. Such interrupt handling is known in the art.

[0066] At block 625, a nonce is sent to the sensor 250. The nonceincludes a time/date stamp, the current session key, and otherinformation. It is used to verify the identity of the sensor as well asthe currency of the fingerprint.

[0067] At block 630, a differential print is received from the sensor250. This, once again, may be an interrupt. The hash is a combination ofthe nonce, and the differential print, as described above. At block 635,the hash is decoded, and the nonce is verified. Additionally, thesession key may be verified.

[0068] At block 640, the differential print is compared to a database oftemplates. The database of templates includes all users who areregistered with this system. The received print is compared to prints inthe database. Such methods are known in the art. Processing continues atblock A shown in FIG. 6B.

[0069] Referring to FIG. 6B, at block 645, the process of the presentinvention tests whether a preliminary match was found. If no match wasfound, the process continues directly to block 670. If the preliminarymatch was found, the process continues to block 650, and both the matchand the hash are returned to the sensor for final matching. This isnecessary if the digital system, in which actual analysis is done, isnot secure. By returning the print and match characteristics to thesensor, the process can be made secure. Alternatively, the final matchmay be done in the digital system 210.

[0070] At block 655, a verifying match/no match signal is received fromthe sensor. Because the sensor is a closed and secure system, the finaldecision, regarding whether a match was found or not, is left to thesensor. In this way, possible tampering with the digital system 210 doesnot result in a false positive signal.

[0071] At block 660, it is determined whether the final answer is a yesor a no, i.e. whether the prints match or do not match. If the prints donot match, at block 670, access is refused.

[0072] If the prints match, at block 665 access to the program, file, orinformation is permitted. Furthermore, if the prints match, at block675, the registered template for the user who was recognized is updated.Updating may be done by executing a matching process in both directions,resulting in a match number, i.e. a number which represents thepercentage of features matched. Based on this number, the best print maybe selected as the new template print. Alternatively, back propagationmay be used to update the current template by adding informationobtained in the new print. Alternatively, a Kahonen type unsupervisedlearning process may be used to update the current template. The Kahonenlearning process is well known to those of skill in the art.

[0073] The entire matching procedure, illustrated in FIGS. 5 and 6 isvery fast. In one embodiment, the matching procedure takes approximatelyhalf a second.

[0074] Security Features

[0075]FIG. 7 is a flowchart illustrating one embodiment of the securityfeatures which may be utilized by the present invention. As discussedabove, because the digital system 210 is not necessarily secure,security procedures are important in order to prevent tampering with thesystem. One of the security procedures is to have the final matching ina secure system. This secure system may be the sensor, which has securedigital keys and data, or in a secure processor. In one embodiment, suchprocessors and chips may be made secure by covering the chip with epoxyor similar hard-to-remove material. Alternatively, in the sensor,security systems may be hard-coded into the system. Furthermore, apublic-key private-key system may be used to make communication betweenthe sensor and any systems used in processing more secure. In oneembodiment, a Diffie-Hellman key exchange using a session key isutilized. Other data encryption methods may be used, and are well knownin the art. Encryption of all data transferred between the sensor andthe digital system, provided by the present invention. Such encryptionallows the present invention to be used in a broader range of non-secureenvironments. FIG. 7 illustrates the Diffie-Hellman key exchange.

[0076] At block 710, the digital system is powered up and the sensor iscoupled to the digital system. At block 720, the digital system requestsa certificate. The certificate is an identifier which is associated witha specific sensor. In one embodiment, the certificate may be stored inthe sensor at the time of manufacture. Each sensor may have a uniquecertificate. Alternatively, multiple sensors may have the samecertificate, if they are associated with the same system. For example, alarge corporation may buy multiple sensors which have the samecertificate, and are thus interchangeable.

[0077] At block 720, the sensor responds with the certificate. Note thatat this point the sensor and digital system are interconnected, buttheir connection is not yet verified to be secure.

[0078] At block 730, the digital system tests the validity of thecertificate using a public key. The sensor has a private key which ispart of the certificate. The public key is used to decode the documentencoded with a private key. If the certificate can not be decoded usingthe known public key, this means that the sensor is not the appropriatesensor, since the certificate was not encoded with the proper privatekey.

[0079] At block 740, the digital system encrypts a random number and atime stamp using the public key and sends it to the sensor. This meansthat only the sensor can decrypt the number. The random number, in oneembodiment, is 56 digits long.

[0080] At block 750, the sensor decrypts the random number and timestamp with the private key. The time stamp is the nonce discussed above.

[0081] At block 760, the random number, decrypted by the sensor, is thesession key used in this session for security. Thus, each time a sessionis established, security is reestablished. Additionally, even if a thiefaccesses the system once, when the session is finished, the thief cannot utilize the data obtained originally to access a second session.

[0082] Sensor and Digital System

[0083]FIG. 8 is a diagram of one embodiment of the sensor of the presentinvention. A central processing unit (CPU) 810 is utilized to interfacewith the universal serial bus (USB) connection 885. The CPU 810 is alsoused to control the power switch 845, and through the power switch, thelight source 840. In one embodiment, the light source 840 is a lightemitting diode (LED). The CPU 810 also controls power to all othercomponents.

[0084] The CPU 810 is coupled to a field programmable gate array (FPGA)815. The FPGA 815 implements most of the functionality of the sensor 250of one embodiment, and may include a random access memory (RAM). TheFPGA 815 and CPU 810 together act as the subtractor, filter, USBinterface, digitizer and security system. A crystal 830 may be coupledto the FPGA 815 to provide a clock signal for the FPGA 815.

[0085] An erasable programmable read only memory (EPROM) 820 is coupledbetween the FPGA 815 and CPU 810. The EPROM 820 contains the informationnecessary for initiating the sensor 250. The EPROM 820 may furthercontain a private key which is associated with that particular sensor250, which permanently tags the sensor's identity.

[0086] A charge-coupled device (CCD) sensor 860 is further coupled tothe FPGA 815. The CCD sensor 860 is a light-sensitive electronic device,which may detect either color or black-and-white. In one embodiment, theCCD sensor 860 is a gray-scale detecting sensor. Each CCD sensor 860consists of an array of light-sensitive photocells. In one embodiment,the photocell is sensitized by giving it an electrical charge prior toexposure. Support electronics 850 is coupled to the CCD sensor 860, toprovide the electronics to activate the CCD sensor 860. The output ofthe CCD sensor 860 is passed through an operation amplifier 870, and alow pass filter 875, to clean up the signal. Then, the cleaned up outputof the CCD sensor 860 is passed through an analog to digital converter880, to digitize the fingerprint. This digitized fingerprint image ispassed on to the FPGA 815.

[0087]FIG. 9 illustrates one embodiment of the configuration of FPGA 815of FIG. 8. Random access memory (RAM) controller 905 controls access toRAM 910. RAM 910 is utilized to store fingerprint images, in opticalform as well as digital form. RAM controller 905 further passesinformation to stability calculator 920. Stability calculator 920 isutilized to calculate when a received fingerprint is stable, asdescribed below.

[0088] RAM controller 905 receives data from analog-to-digital converter880. RAM controller 905 is clocked by a clock received through a timingcontrol 945. FPGA 815 further includes a multiplexer (MUX) 915. MUX 915is used to access address and data lines in the CPU 810, as shown inFIG. 8.

[0089] RAM controller 905 includes a subtractor 990, which is used tosubtract a background image from a fingerprint, to determine adifferential print. The output of the subtractor is coupled to thestability calculator 920. The output of the stability calculator 920 iscoupled to a stability data register 970, which is a status register,indicating when the fingerprint image is stable.

[0090] Compressor 925 compresses the fingerprint image prior totransmitting it through the parallel port interface 930. Alternatively,the compressed image may be transmitted via the USB interface 935. Inone embodiment, compressor 925 utilizes Huffman compression, which is aconventional data compression technique which varies the length of theencoded symbol in proportion to its information content, i.e. the moreoften a symbol or token is used, the shorter the binary string used torepresent it in the compressed stream.

[0091] General peripheral interface, or universal serial bus interface935 is utilized to pass data to the universal serial bus. In oneembodiment, the USB access is accomplished through the CPU 810. However,the USB interface 935 may be located in the FPGA 815.

[0092] A status register 940 is coupled to the MUX 915 and the CPUinterface. The status register 940 indicates the general status of thecommunications devices, including the parallel port, USB port, andframes. Status register 940 is controlled by the CPU 810, and passes itsinformation to the RAM controller 905.

[0093] A control register 950 is also coupled to the MUX 915. Controlregister is set by the CPU 810 to control the functions of the FPGA 815.Address register 955 contains the address of the RAM data in RAM 910,allowing CPU 810 access to the RAM data. In one embodiment, addressregister 955 is automatically incremented when data has been written toan address. Address register is controlled by control register 950. Readdata register 965 and write data register 960 buffer data read from orwritten to RAM 910.

[0094] Threshold register 975 includes the threshold numbers needed forstability calculations by stability calculator 920. These thresholdnumbers are described in more detail below. In one embodiment, thresholdregister 975 may be written to by the CPU 810. Port data register 985 isa register for data sent from the CPU 810 to be sent over the parallelport interface. The USB data register 980 is a register of data from theUSB interface. In one embodiment, the USB data register is a read onlyregister, which stores system information.

[0095]FIG. 10 is a diagram of one embodiment of the digital system ofthe present invention. Digital system 1000 comprises a system bus 1010or other communication means for communicating information, and aprocessor 1020 coupled with system bus 1010 for processing information.Digital system 1000 also comprises a read only memory (ROM) and/or otherstatic storage device 1035 coupled to system bus 1010 for storing staticinformation and instructions for processor 1020. The digital system 1000further comprises a main memory 1030, a dynamic storage device forstoring information and instructions to be executed. Main memory 1030also may be used for storing temporary variables or other intermediateinformation during execution of instructions. In one embodiment the mainmemory 1030 is dynamic random access memory (DRAM).

[0096] Digital system 1000 further comprises a universal serial bus(USB) controller 1080, a bus controller for controlling a universalserial bus (USB) 1085. The USB 1085 is for coupling USB devices 1090 tothe digital system 1000. The sensor 250 may be one of the USB devices1090 coupled to the digital system 1000 via the USB 1085.

[0097] Digital system 1000 can also be coupled via system bus 1010 to adisplay device 1050, such as a cathode ray tube (CRT) or liquid crystaldisplay (LCD) screen, for displaying information to a user. Analphanumeric input device 1055 is typically coupled to system bus 1010for communicating information and command selections to processor 1020.Another type of user input device is cursor control device 1060, such asa mouse, a trackball, trackpad, or cursor direction keys forcommunicating direction information and command selections to processor1020 and for controlling cursor movement on display device 1050.Alternatively, other input devices such as a stylus or pen can be usedto interact with the display. The digital system 1000 may further becoupled via the system bus 1010 to a network communication device 1065.The network communication device 1065 may be utilized to couple thedigital system to other digital systems, servers, and networks.

[0098] Registration

[0099]FIGS. 11A and 11B are a flowchart illustrating the process ofregistering a fingerprint. The registering is initiated by the useropening a registering interface. Alternatively, when the present systemis first initiated, the registration process may automatically open.

[0100] At block 1110, an identity of a person trying to register isrequested. The identity may be initials, or some other identifier whichmay later be associated with the fingerprint. In one embodiment, thefull name of the user is requested here. Alternatively, for a wallet, orsimilar “owned” system, no identity is requested, and it is assumed thatthe individual who is registered with this process is the owner of theitem in question.

[0101] At block 1115, a fingerprint is requested. This may be done bydisplaying a window, with text such as “please place your finger on thesensor now for registering”, or similar text. Alternatively, the systemmay just wait for a fingerprint, without displaying a request.

[0102] At block 1120, a number of different prints are taken. Asdescribed above, the CCD sensor 860 detects the fingerprint, and copiesthe optical image to a RAM. In this instance, a plurality of prints maybe taken, even if the user places his or her finger on the sensor platenonce.

[0103] At block 1125, each of the fingerprints are digitized. Theseprints are temporarily stored in the sensor. Alternatively, they may bestored in the digital system.

[0104] At block 1130, a digitized background image is subtracted fromeach of the digitized prints, resulting in a differential prints. Asdescribed above, a digitized background image is obtained when thesensor is initially turned on, and every time after a fingerprint isobtained.

[0105] At block 1135, the prints are compared. This may be done usingknown techniques of evaluating fingerprints. For example, three or moreprints may be matched, finding out how many matched points exist betweenprints. Such a comparison will determine which print is the bestquality.

[0106] At block 1140, the best print is selected. Because theregistration print is one to which later fingerprints are compared, itshould be the best print possible. Therefore, the print which displaysthe fingerprint characteristics most clearly is selected. Alternatively,information from all the registration prints may be assembled into asingle composite print which is stored in the database. Alternatively,in addition to selecting the best print, the best print is furtherimproved, using a learning process learning from the other prints.Furthermore, as described above with respect to block 675, the templatemay be updated every time a new print is received from the user. Thisprocess is known in the art. Alternatively, this step may be skipped.

[0107] At block 1145, shown in FIG. 11B, the differential print isstored in the database. Alternatively, this selection process may beskipped, and all of the prints may be stored in the database as aregistration dataset.

[0108] At block 1150, the system requests any files and applicationswhich the user wishes to associate with the fingerprint and identity.The user can associate files and applications which can be openedautomatically after a successful fingerprint validation process. If nosuch files are associated, the fingerprint merely validates the user, orpermits the user access to the user's own data stored in the digitalsystem 210 or any other system coupled to the digital system 210.

[0109] At block 1155, it is determined whether any files were identifiedby the user. If no files are identified, the process continues to block1180. If files were identified, the process continues to block 1160.

[0110] At block 1160, the system determines what kind of security thereis on the file selected. The user can select program files, applicationfiles, or files such as specific data files, or word processor files.Some files may be already password protected. For example, Word forWindows permits password protection of a data file. Word for Windows isa trademark of Microsoft Corporation. If the file selected either has nosecurity or easily alterable security, the process continues to block1165.

[0111] At block 1165, the boot sector of the file selected is altered topermit fingerprint verification or identification. Thus, when the userwishes to access that file, the user will have to show his or herfingerprint in order to access the file. Alterations of a boot sectorare known in the art. The format of the boot sector is platformdependent, and thus is known and alterable for each platform. Therefore,the process accesses the boot sector of the file selected, and altersit. In one embodiment, this alteration consists of pointing the bootprocess to a fingerprint identification subroutine. The end of thefingerprint identification subroutine points back to the boot sector.Thus, during execution, the fingerprint verification subroutine iscalled up and executed.

[0112] At block 1160, if the file selected is already passwordprotected, and such protection is not easily alterable, the processcontinues to block 1170. At block 1170, the password associated withthat password protected file is requested. The user has to enter thepassword associated with the file. And, at block 1175, the passwordassociated with that file is stored in the database, associated with thefingerprint and identity of the user. Thus, next time the user selectsthe file, and uses the fingerprint, the system automatically inserts thepassword associated with that file and individual, and opens the file.

[0113] At block 1155, if no files were listed, the process continues toblock 1180. At block 1180, any default access is identified, andassociated with the fingerprint. Default access may be provided, forexample, to permit access to a special area in a server or system whichstores the user's personal files. Thus, for example, fingerprintidentification may provide automatic access to a special files list. Inone embodiment, the user may define such an area. Alternatively, theuser identification, entered at block 1110, may be used to determine thedefault area access. Alternatively, if no such area exists, the processcontinues directly to block 1185. At block 1185, the registrationprocess is completed.

[0114] Capturing Fingerprint

[0115]FIG. 12A is a flowchart illustrating the process of capturing afingerprint image. This is a more detailed illustration of thefunctionality described with respect to blocks 535, 540, and 545 in FIG.5, i.e. capturing fingerprint, digitizing print, and subtracting thebackground from the print.

[0116] At block 1200, the presence of an initial print is detected. Asdiscussed above, this may be detected as a result of the change in theenergy hitting the sensor platen of the sensor. At block 1205, a sum isset to zero. The use of this sum will become apparent below.

[0117] At block 1210, a print is captured and digitized. This process isdescribed above in more detail. At block 1215, a digitized backgroundimage is subtracted from the digitized print. This results in a moreclear representation of the actual print on the sensor platen.

[0118] At block 1220, the image is filtered, by taking every Xth row andYth column, and taking only those pixels which are in those columns androws. This reduces the further processing considerably, since only1X*Yth as many pixels need to be evaluated.

[0119]FIG. 12B illustrates this filtering process. Picture A is theimage with all of the pixels, represented by small boxes. In reality,since this is a digitized image, each of those boxes corresponds to anintensity factor of that particular pixel. Picture B represents theimage showing how the 0th and 3rd column and row of the pixels areselected. And Picture C represents the filtered image, with only thosepixels which are in the filtered picture represented by boxes. In oneembodiment, X and Y are both four. Thus, only 1 of 16 pixels areevaluated. Further processing is executed on the filtered image. Thisblock may be skipped in some embodiments.

[0120] Returning to FIG. 12A, at block 1225, the sum of the intensity ofpixels above the threshold is determined for the filtered image. In oneembodiment, the digitized figure is a gray scale figure, which meansthat pixels may have a variety of intensities. In one embodiment, pixelintensity may vary between 0 and 6. The threshold in this instance isthe average intensity of energy of a fingerprint. Thus, for example, thethreshold may be 3. For three pixels which are 2, 4, and 6, the resultswould be 0, +1, +3, thus the sum would be four (4).

[0121] At block 1230, the derivative of this sum is taken. Thisdetermines the rate of growth of the sum, compared to the prior sum. Inone embodiment, the derivative is determined by subtracting the priorsum from the current sum.

[0122] At block 1235, it is determined whether the derivative is below acertain threshold. This threshold may be determined based on the generalfingerprint quality of the sensor. If the derivative is not below thethreshold, in other words, the sum is still growing, the process returnsto block 1210, and the next image is captured. If the derivative isbelow the threshold, the process continues to block 1240.

[0123] At block 1240, it is determined whether the sum is above acertain minimum threshold. This threshold is the average contrast ridgeto valley for a fingerprint taken with that particular sensor. In oneembodiment, this threshold may be updated in the system based on laterresults. If the sum is not above the threshold, the process returns toblock 1210, and the next image is taken. If the sum is above the minimumthreshold, the fingerprint is sufficiently large and intense, and theprocess continues to block 1245 and block 1250. At block 1245, the finalprint is captured, and passed on for further processing.

[0124]FIG. 12C represents an illustration of this process. For example,Picture D represents an image which is barely visible. Because the firstimage is captured when the energy level first decreases, it is only apartial picture of the finger. Pictures D, E, F, G, and H show thegradual increase of the size of the fingerprint, during successivepictures which are captured. When Picture I is captured, the systemdetermines that the size is decreasing. The fingerprint size representedby the previous image, Picture H, is therefore captured and retained asthe optimal fingerprint image.

[0125] Returning to FIG. 12A, at block 1250, the sum is reset to zero.At block 1255, the next image is captured and digitized. The backgroundis subtracted from the image at block 1260.

[0126] At block 1265, the image is filtered, and at block 1270, the sumof the intensity of the pixels above a threshold intensity are added. Aderivative of this sum is taken at block 1275. At block 1280, it isdetermined whether the derivative is below a threshold. This thresholdis similar to the threshold discussed above with respect to FIG. 1235.When the images stop decreasing, the derivative of the images getscloser to zero, and this derivative meets the threshold. If thederivative is not below the threshold, the process returns to block1255. If the derivative is below the threshold, the process continues toblock 1285. At block 1285, it is determined whether the sum is below acertain threshold. If the sum is not below the threshold, the processreturns to block 1255. If the sum is below the threshold, the processcontinues to block 1290. At block 1290, the new background is captured.As can be seen, blocks 1255 to 1285 are the reverse of blocks 1210 to1240. Thus, the process is executed in reverse to obtain a newbackground image. This background image is used in the next process.

[0127] Auto-launch Feature

[0128]FIG. 13 is a flowchart illustrating the process of auto-launchingan application or document. At block 1310, the application is initiated.The application may be initiated by the user selecting the file, by theuser placing his or her fingerprint on the sensor, or other means.

[0129] At block 1315, it is determined whether there is a fileassociated with the fingerprint in the database as described earlier. Inother words, if the file which was initiated does not have an associatedfingerprint verification/identification requirement, the processcontinues to block 1320, where the auto-launch process aborts. If theuser selected a file which does have fingerprintverification/identification, the process continues to block 1325.

[0130] At block 1325, the process waits for a fingerprint. The processmay display a note, requesting a fingerprint. If the user initiated theprocess by placing his or her finger on the sensor, the process goesdirectly to this block. In one embodiment, the process starts here, andblocks 1310 and 1315 are skipped. This may be applicable when a userwishes to access the user's special storage area, or a default fileusing fingerprint recognition.

[0131] At block 1330, the fingerprint is processed to recognition. Thisprocess is as described above.

[0132] At block 1335, the database is queried to determine if the useris authorized to access the selected file or application. The user'sidentification is determined at block 1330, thus, this can be automated.If the user is not authorized to access the selected file, the processcontinues to block 1340, and access to the file is denied. In oneembodiment, the system displays the message “User not authorized toaccess selected file”, or similar message.

[0133] If the user is authorized to access the selected file, theprocess continues to block 1345. At block 1345, the file/application isinitialized. If this is an application program, the application programis selected. If it is a file, to be opened by an application program,the file is found, and selected.

[0134] At block 1350, it is determined what type of auto-launch isassociated with the selected file/application. As discussed above withrespect to the registration, FIG. 11, the file may be either boot strapauto-launched or password auto-launched.

[0135] Returning to FIG. 13, at block 1350, if the process is passwordauto-launch, the process continues to block 1355. At block 1360, thepassword associated with the file and the fingerprint is looked up inthe database. As described above, in FIG. 11, for password enabledfiles/applications the password is stored in the database. At block1365, the password is inserted into the file, in the proper location,and the file is opened/accessed.

[0136] At block 1350, if the process is direct auto-launch, the processcontinues to block 1370. At block 1370, the bootstrap is used to run theapplication or access to the file. As described above with respect toFIG. 11, the boot sector of the file may be altered to enablefingerprint access. Thus, when the fingerprint is received thefile/application is automatically called up.

[0137] Token Interface

[0138]FIG. 14 is a flowchart illustrating the process of using a tokenin conjunction with the fingerprint recognition system of the presentinvention. The token may be a key, bar code, diskette, smart card orsimilar external data container.

[0139] At block 1410, the user inserts the token. In one embodiment, thetoken needs to be coupled to the digital system. In an alternativeembodiment, the digital system may be incorporated within the token, andthus the token need only be coupled to the sensor.

[0140] At block 1415, the process tests whether fingerprint validationis enabled. The token may be usable without fingerprint validation. Iffingerprint validation is not enabled, the process continues to block1420. At block 1420, the process terminates, since no fingerprintvalidation is enabled. If fingerprint validation is enabled, the processcontinues to block 1425.

[0141] At block 1425, a fingerprint is requested. This may occur by thesystem displaying a message such as “please place your finger on thesensor”, or alternatively may just be an internal wait state forfingerprint validation. Thus, here the user places his or her finger onthe sensor.

[0142] At block 1430, the validity of the template on the token isconfirmed. The token contains a template of the authorized user'sfingerprint. The process tests whether the template has been tamperedwith. In one embodiment, a public-key private-key signature exchange isused to verify that the template is secure.

[0143] At block 1435, the fingerprint is compared with a templates onthe token. The token is “owned” by one or more users. At this stage, theprocess tests whether a user who an owner of the token is using it.

[0144] At block 1445, The process test whether the fingerprint is aprint of an owner of the token. The token's owner's prints areregistered within the token. If the fingerprint is not the print of thetoken owner, the process continues to block 1450, and the token isdisabled. Furthermore, an alert may be sent out indicating that thetoken is being used by an unauthorized user.

[0145] If the fingerprint is the print of the token owner, the processcontinues to block 1455. At block 1455, the token is enabled, and theuser can access data stored on the token. For example, if a user owns asmart card which is a credit card, this process may be used to verifythat the smart card is not stolen, and actually belongs to the user inquestion. A thief could not activate the smart card, and thus get accessto the confidential information stored on the card.

[0146] In the foregoing specification, the invention has been describedwith reference to specific embodiments thereof. It will, however, beevident that various modifications and changes may be made theretowithout departing from the broader spirit and scope of the invention.The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. The present inventionshould not be construed as limited by such embodiments and examples, butrather construed according to the following claims.

What is claimed is:
 1. A method of recognizing a fingerprint comprising:detecting the fingerprint; digitizing the fingerprint; subtracting adigitized background from the fingerprint, resulting in a differenceprint; identifying an individual associated with the difference print.2. The method of claim 1 further comprising: detecting a background; anddigitizing the background, resulting in the digitized background.
 3. Themethod of claim 1 further comprising: initializing a fingerprintrecognition system when the fingerprint recognition system is initiallyturned on; detecting a first background when the fingerprint recognitionsystem is being initialized; digitizing the first background; anddetecting a new background after the fingerprint is detected; anddigitizing the new background.